Generating dolly zoom effect using light field image data

ABSTRACT

A sensor is configured to acquire a light field by imaging a scene. A processor is configured to determine four-dimensional (4D) coordinates of points in a light field and generate dollied coordinates from the 4D coordinates based on a dolly transform and a dolly parameter. The processor is also configured to project rays associated with the dollied coordinates from the light field onto corresponding points in an output raster. In some cases, the processor applies an aperture function to filter the rays in the coordinate system of the dollied coordinates. The aperture function has a first value in a first region of an aperture plane and the aperture value has a second value in a second region of the aperture plane. Rays passing through the first region are accepted by the aperture function and rays passing through the second region are rejected.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority to U.S. Provisional ApplicationSer. No. 62/481,038 for “Generating Dolly Zoom Effect Using Light FieldImage Data” (Atty. Docket No. LYT274-PROV), filed on Apr. 3, 2017, whichis incorporated herein by reference.

The present application is related to U.S. Utility Application Ser. No.14/311,592 for “Generating Dolly Zoom Effect Using Light Field ImageData” (Atty. Docket No. LYT003-CONT), filed on Jun. 23, 2014, and issuedon Mar. 3, 2015 as U.S. Pat. No. 8,971,625, which is incorporated hereinby reference.

The present application is related to U.S. Utility Application Ser. No.15/162,426 for “Phase Detection Autofocus Using Subaperture Images”(Atty. Docket No. LYT225), filed on May 23, 2016, which is incorporatedherein by reference.

BACKGROUND

A light field camera, also known as a plenoptic camera, capturesinformation about the light field emanating from a scene; that is, theintensity of light in a scene, and also the direction that the lightrays are traveling in space. This contrasts with a conventional camera,which records only light intensity. One type of light field camera usesan array of micro-lenses placed in front of an otherwise conventionalimage sensor to sense intensity, color, and directional information.Multi-camera arrays are another type of light field camera.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure may be better understood, and its numerousfeatures and advantages made apparent to those skilled in the art byreferencing the accompanying drawings. The use of the same referencesymbols in different drawings indicates similar or identical items

FIG. 1 illustrates an example of a dolly zoom effect according to someembodiments.

FIG. 2 illustrates an example of dolly-zoom as a coordinatetransformation according to some embodiments.

FIG. 3 is an illustration of a dolly-zoom as a transformation of thelight field according to some embodiments.

FIG. 4 illustrates a “dolly twirl” analogous to the dolly zoom of FIG. 2according to some embodiments.

FIGS. 5 and 6 show examples of opposite extremes of a dolly-zoom actionaccording to some embodiments.

FIGS. 7 and 8 show examples of opposite extremes of a dolly-twirl actionaccording to some embodiments.

FIG. 9 is a block diagram of an architecture of a light field capturedevice such as a camera according to some embodiments.

FIG. 10 is a block diagram of an architecture of a light field capturesystem according to some embodiments.

DETAILED DESCRIPTION

Light fields can be manipulated to computationally reproduce variouselements of image capture. One example is refocusing, which isequivalent to a virtual motion of the sensor plane. As described herein,virtual motion of the main lens plane is also possible, resulting in adolly zoom effect in which a field-of-view is modified while a camellocation moves towards or away from a subject in a manner that maintainsa size of the subject in the frame. The dolly zoom effect was first usedto portray James Stewart's fear of heights in the film Vertigo and isalso well known for its use portraying Roy Scheider's anxiety in thefilm Jaws. Existing techniques to create a dolly zoom effect rely on theuse of a depth map during 2D reconstruction. This requirement of anaccurate depth map limits the robustness and applicability of thedolly-zoom effect.

Some embodiments of the light field are represented as afour-dimensional (4D) function that can be formed of two-dimensional(2D) images of a portion of a scene. The images represent views of theportions of the scene from different perspectives and frames arerendered from the point of view of the camera by sampling portions ofthe 2D images. For example, the coordinates (u, v, s, t) in the 4Dfunction that represents the light field can be defined so that (u, v)represent coordinates within one of the camera images in the light fieldand (s, t) are coordinates of pixels within the camera image. Otherdefinitions of the four coordinates of the 4D function that representsthe light field can also be used.

According to various embodiments, a dolly zoom effect can be applieddirectly to the light field as a 4D transformation of some or all of thecoordinates (u, v, s, t) that are used to define locations in the 4Dlight field. Using the techniques described herein, explicit depthcalculations are not required, allowing the approach to be robustlyapplied to any light field. Furthermore, the techniques described hereindo not require 2D reconstruction, thus allowing the full 4D light fieldto be dollied, for example for applications such as light fielddisplays.

The described techniques can be implemented as a 4D transformation ofthe light field, and can easily be combined with other transformations,such as refocus, to create various effects.

Basic Dolly Zoom Equation

A simple version of the dolly zoom equation can be derived analogouslyto the refocus equation. For this derivation we reduce the light fieldto 2D without loss of generality.

FIG. 1 illustrates an example 100 of a dolly zoom effect according tosome embodiments. The s coordinate denotes position on the sensor 105,while the u coordinate denotes position on the aperture 110. Three rays111, 112, 113 (collectively referred to herein as “the rays 111-113”)are shown to pass through a location on the aperture 110 (notnecessarily at the origin of the coordinate system used in the plane ofthe aperture 110) and hit the sensor 105.

A virtual aperture 120 is dollied a distance d relative to the plane ofthe aperture 110. From simple geometry it is apparent that the locationswhere the rays 111-113 intersect the virtual aperture 120 changeproportionally to the slope of the rays 111-113 and d. Thus, thelocation where one of the rays 111-113 intersects the virtual aperture120 is:

u′=u+k·s·d

where k is a normalization constant such that k*s is the slope of theray.

In some embodiments, the values of k and d our combined into a singleconstant γ.

Thus, the light field transformation can be expressed as:

$\begin{bmatrix}s^{\prime} \\u^{\prime \;}\end{bmatrix} = {\begin{bmatrix}1 & 0 \\\gamma & 1\end{bmatrix}\begin{bmatrix}s \\u\end{bmatrix}}$

or for the full 4D transform the light field is represented as:

$\begin{bmatrix}s^{\prime} \\t^{\prime} \\u^{\prime} \\v^{\prime \;}\end{bmatrix} = {\begin{bmatrix}1 & 0 & 0 & 0 \\0 & 1 & 0 & 0 \\\gamma & 0 & 1 & 0 \\0 & \gamma & 0 & 1\end{bmatrix}\begin{bmatrix}s \\t \\u \\v\end{bmatrix}}$

Mathematically this transformation skews the light field along the uvplane. Intuitively, this skewing of the light field along the uv planecan be thought of as introducing a perspective shift that isproportional to the st location. An object's motion with a perspectiveshift is proportional to its depth, which can be interpreted as a depthdependent scaling. However, there is no need for the depth to beexplicitly calculated.

Thinking of this dolly action as a depth-dependent scaling reveals that,for objects at the focus position (lambda 0), no scaling occurs. Thiscan be thought of as a counter zoom being applied in order to keepapparent size the same for objects at the focus distance. Thus thistransform simultaneously dollies and zooms the light field image.

Example Implementation

In at least one embodiment, a dolly zoom effect is implemented using thelight field projection method of 2D reconstruction as follows:

-   -   1. A 4D coordinate (s, t, u, v) is accessed or calculated for        each point in the light field.    -   2. Given a dolly parameter γ, the dollied coordinate (s′, t′,        u′, v′) is determined using the above equation, i.e. s′=s, t′=t,        u′=u+γs, v′=v+γt    -   3. An aperture function a (u, v) is accessed or generated. In        the illustrated embodiment, the aperture function has a value of        1 where the aperture accepts rays and 0 where the aperture        rejects rays. However, in other embodiments, the aperture        function is implemented using other values or methods of        representation.        -   a. Some embodiments of a circular aperture with radius r            have a value of the aperture function a(u, v) equal to 1            when u²+v²<r² and 0 otherwise. The natural fully open            aperture would have r=0.5        -   b. The transformation of u′ and v′ effectively shifts the            center of the aperture. A somewhat smaller aperture than            full open can be used, so as to avoid the aperture function            going out of bounds of the actual data, as shown in FIG. 2.        -   4. If the aperture function a(u′, v′) accepts the ray, then            the light field ray is projected to point (x, y)=(s′, t′) on            the output raster.

FIG. 2 illustrates an example of dolly-zoom as a coordinatetransformation according to some embodiments. From left to right, thereis shown: the original light field 200, the light field with negativedolly-zoom 205, and the light field with positive dolly-zoom 210. Thecenter (uv) location of each st cell changes.

In the example of FIG. 2, the above-described method is illustratedusing a 3×3 light field. Each cell represents an (s, t) coordinate,while the coordinate within each cell represents (u, v). The centralpoint 215 (only one indicated by a reference numeral in the interest ofclarity) within each cell is the center of the (u, v) coordinate system.The gray areas represent valid data within the light field. The circle220 (only one indicated by a reference numeral in the interest ofclarity) represents the aperture function that is used in the abovealgorithm, which as noted above is smaller than the full size of thecaptured data.

The illustrations are examples only, and do not imply any particularmethod of light field capture. However, in some embodiments, a plenopticcamera with a 3×3 microlens array is used to capture the light field. Insuch a case, each microlens would be the (s, t) location, and withineach microlens is a (u, v) coordinate system centered on the red x.

FIG. 2 illustrates an intuitive interpretation that dolly-zoom producesa spatially varying perspective shift. The virtual aperture, a.k.a. theaperture function, undergoes a perspective shift that is proportional toits (s, t) location, thus producing depth-dependent scaling in the 2Dreconstructed image. This can be visualized as the scaling of the gridformed by the central points 215. As mentioned above, since the virtualaperture is being shifted, the virtual aperture is oriented or sized sothat it overlaps with valid data. This also means a smaller virtualaperture will allow for more dolly-zoom.

FIG. 3 is an illustration of a dolly-zoom as a transformation of thelight field according to some embodiments. From left to right, there isshown: the original light field 300, the light field with negativedolly-zoom 305, and the light field with positive dolly-zoom 310. Thecoordinate system is the same throughout, but the underlying light fieldis transformed. The central point 315 (only one indicated by a referencenumeral in the interest of clarity) within each cell is the center ofthe (u, v) coordinate system. The gray areas represent valid data withinthe light field. The circle 320 (only one indicated by a referencenumeral in the interest of clarity) represents the aperture functionthat could be used in the above algorithm, which as noted above issmaller than the full size of the captured data.

FIG. 3 thus illustrates the mathematical equivalent of performing atransformation on the underlying light field data (perhaps using somesort of interpolation) instead of shifting the virtual apertures.

None of the above is to imply that an aperture function is needed orthat 2D reconstruction is necessary. The use of the aperture function inthe above figures is merely to illustrate the effect the dolly-zoomwould have in the example 2D reconstruction algorithm. Other 2Dreconstruction methods may be used that do not utilize an aperture, and2D reconstruction is not even necessary if method is applied as a 4Dtransform of the light field. The 4D light field itself may be modified,either by changing the coordinate mapping as in FIG. 2, or bytransformation of the underlying data as in FIG. 3, and then stored forfuture processing.

FIGS. 2 and 3 illustrate two different approaches that are useful indifferent situations. The first approach (FIG. 2) is useful in caseswhere light field data is irregularly gridded, or in other cases whereexplicit coordinates of each light field point are known and/orpreferred. The modified light field coordinates are then fed todownstream processing, such as for example projection for 2Dreconstruction. The second approach (FIG. 3) is useful in cases of wherelight field data is regularly gridded, or in other cases where the lightfield coordinates are implicitly known. The underlying data is modifiedand passed onto downstream processing. This can be useful in the case ofa light field display, for example. In the second case, the data may bemodified such that the data ends up outside of the original implicitgrid, as shown in FIG. 3. This can be compensated for by padding theoriginal data before transformation.

Composing Multiple Transforms

For illustrative purposes, and without loss of generality, the techniqueof combining multiple transforms is described in terms of a 2D lightfield.

Since dolly zoom is mathematically a linear transform, it can becombined with other linear transforms by matrix multiplication. Forexample, refocus and dolly zoom operations can be combined by using oneof two transforms:

$\begin{bmatrix}s^{\prime} \\u^{\prime \;}\end{bmatrix} = {{{\begin{bmatrix}1 & \lambda \\0 & 1\end{bmatrix}\begin{bmatrix}1 & 0 \\\gamma & 1\end{bmatrix}}\begin{bmatrix}s \\u\end{bmatrix}} = {{{\begin{bmatrix}{1 + {\lambda \; \gamma}} & \lambda \\\gamma & 1\end{bmatrix}\begin{bmatrix}s \\u\end{bmatrix}}\begin{bmatrix}s^{\prime} \\u^{\prime \;}\end{bmatrix}} = {{{\begin{bmatrix}1 & 0 \\\gamma & 1\end{bmatrix}\begin{bmatrix}1 & \lambda \\0 & 1\end{bmatrix}}\begin{bmatrix}s \\u\end{bmatrix}} = {\begin{bmatrix}1 & \lambda \\\gamma & {1 + {\lambda \; \gamma}}\end{bmatrix}\begin{bmatrix}s \\u\end{bmatrix}}}}}$

The two transformations are not equivalent because composingtransformations is not a commutative operation. Depending on the effectone wishes to achieve, the order of composition must be carefullyselected.

In order to implement this in the previous example algorithm, step 2 canbe modified in one of two ways.

Dolly First: 2.1 First modify the (u, v) coordinates as u′ = u + γs, v′= v + γt 2.2 Then modify the (s, t) coordinates as s′ = s + λu′, t′ =t + λv′ 2.2.1 Alternatively, modify the (s, t) coordinates at the sametime as 2.1, as per the above equations s′ = s(1 + λγ) + λu, t′ = t(1 +λγ) + λv Refocus First: 2.1 First modify the (s, t) coordinates as s′ =s + λu, t′ = t + λv 2.2 Then modify the (u, v) coordinates as u′ = u +γs′, v′ = v + γt′ 2.2.1 Alternatively, modify the (u, v) coordinates atthe same time as 2.1, as per the above equations u′ = u(1 + λγ) + γs, v′= v(1 + λγ) + γt

One might be tempted to try to refocus and dolly simultaneously by usingthe following transform:

$\begin{bmatrix}s^{\prime} \\u^{\prime \;}\end{bmatrix} = {\begin{bmatrix}1 & \lambda \\\gamma & 1\end{bmatrix}\begin{bmatrix}s \\u\end{bmatrix}}$

However this is not a valid coordinate transform. In particular it ispossible to select λ and γ such that the matrix is singular and thus notinvertible.

More complicated compositions can be used as well. For example:

$\begin{bmatrix}s^{\prime} \\u^{\prime \;}\end{bmatrix} = {{{\begin{bmatrix}1 & \lambda_{2} \\0 & 1\end{bmatrix}\begin{bmatrix}1 & 0 \\\gamma & 1\end{bmatrix}}\begin{bmatrix}1 & \lambda_{1} \\0 & 1\end{bmatrix}}\begin{bmatrix}s \\u\end{bmatrix}}$

One use of this composition is to execute a dolly zoom where theconstant-size depth is at a different depth than lambda zero. In thiscase, the first refocus effectively changes the zero lambda depth, thedolly zoom occurs, and then the second refocus undoes the first refocus.The second refocus is not simply the inverse of the first refocus (sincethe dolly action remaps the refocus depths), but this could be a goodfirst approximation. Exact relations may be derived so to have therefocuses exactly cancel out.

Certain compositions of transforms may be expressed as other transformsas well. In some embodiments, the above composition is utilized with thevalues:

$\lambda_{1} = {\lambda_{2} = {- {\tan \left( \frac{\theta}{2} \right)}}}$γ = sin (θ)

In that case, the transformation is equivalent to rotating the lightfield in the epipolar plane

$\begin{bmatrix}s^{\prime} \\u^{\prime \;}\end{bmatrix} = {\begin{bmatrix}{\cos \; \theta} & {\sin \; \theta} \\{{- \sin}\; \theta} & {\cos \; \theta}\end{bmatrix}\begin{bmatrix}s \\u\end{bmatrix}}$

Rotation by shearing is described, for example, in A. W. Paeth, A FastAlgorithm for General Raster Rotation, Computer Graphics Laboratory,Department of Computer Science, University of Waterloo, 1986.

Spatially Varying Transform

Neither the refocus parameter λ nor the dolly zoom parameter γ isrequired to be constant. Some embodiments of the refocus parameter λ orthe dolly zoom parameter γ vary as a function of (s, t, u, v), or even(s′, t′, u′, v′). For example, the dolly zoom parameter γ can vary as afunction of (u, v) and the refocus parameter λ can vary as a function of(s, t). If γ(u,v) is a planar function, this mimics the physical actionof tilting the lens. This is analogous to how sensor tilt can bemimicked by having λ(s, t) be a planar function.

One artistic effect is called “Lens Whacking”, which is achieved byshooting with the lens free floating from the body. This means thatfocus and tilt effects are achieved by the relative placement of thelens to the sensor. By utilizing the ability to virtually produce tiltin both the lens plane and aperture plane, lens whacking effects can beperformed computationally in a light field.

Depth-Based Transformations

In at least one embodiment, the dolly zoom effect represents adepth-dependent scaling of the image. Thus, many 2D transforms can begeneralized to be depth dependent. One possible way for lineartransforms is to use the equation

$\begin{bmatrix}s^{\prime} \\t^{\prime} \\u^{\prime} \\v^{\prime \;}\end{bmatrix} = {{\begin{bmatrix}1 & 0 & 0 & 0 \\0 & 1 & 0 & 0 \\{t_{11} - 1} & t_{12} & 1 & 0 \\t_{21} & {t_{22} - 1} & 0 & 1\end{bmatrix}\begin{bmatrix}s \\t \\u \\v\end{bmatrix}} = {\begin{bmatrix}I & 0 \\{T - I} & I\end{bmatrix}\begin{bmatrix}s \\t \\u \\v\end{bmatrix}}}$

Where the linear transform is described by

$T = \begin{bmatrix}t_{11} & t_{12} \\t_{21} & t_{22}\end{bmatrix}$

In this simple case where T is a scaling matrix,

$T = {\left. \begin{bmatrix}s & 0 \\0 & s\end{bmatrix}\Rightarrow{T - I} \right. = \begin{bmatrix}{s - 1} & 0 \\0 & {s - 1}\end{bmatrix}}$

It is apparent that this reduces to the dolly zoom equation in the casein which γ=s−1.

Another potentially interesting effect is rotation. In this case

$T = {\left. \begin{bmatrix}{\cos \; \theta} & {\sin \; \theta} \\{{- \sin}\; \theta} & {\cos \; \theta}\end{bmatrix}\Rightarrow{T - I} \right. = \begin{bmatrix}{{\cos \; \theta} - 1} & {\sin \; \theta} \\{{- \sin}\; \theta} & {{\cos \; \theta} - 1}\end{bmatrix}}$

Using a small angle approximation this can be further simplified to

$\begin{bmatrix}{{\cos \; \theta} - 1} & {\sin \; \theta} \\{{- \sin}\; \theta} & {{{\cos \; \theta} - 1}\;}\end{bmatrix} \approx \begin{bmatrix}0 & \theta \\{- \theta} & 0\end{bmatrix}$

Thus, the following transform produces a rotation in the imageproportional to depth (“dolly twirl”):

$\begin{bmatrix}s^{\prime} \\t^{\prime} \\u^{\prime} \\v^{\prime \;}\end{bmatrix} = {\begin{bmatrix}1 & 0 & 0 & 0 \\0 & 1 & 0 & 0 \\0 & \theta & 1 & 0 \\{- \theta} & 0 & 0 & 1\end{bmatrix}\begin{bmatrix}s \\t \\u \\v\end{bmatrix}}$

FIG. 4 illustrates a “dolly twirl” analogous to the dolly zoom of FIG. 2according to some embodiments. From left to right there is shown: theoriginal light field 400, the light field with negative theta 405, andthe light field with positive theta 410. The central point 415 (only oneindicated by a reference numeral in the interest of clarity) within eachcell is the center of the (u, v) coordinate system. The gray areasrepresent valid data within the light field. The circle 420 (only oneindicated by a reference numeral in the interest of clarity) representsthe aperture function that could be used in the above algorithm, whichas noted above is smaller than the full size of the captured data. Thecenter (uv) location 415 of each st cell changes.

This technique can be further generalized to higher order transforms.For any given 2D transform, the identity part can be removed, and theremainder applied to the uv coordinates. This can be used to createeffects that are otherwise not possible to produce using traditionalcamera setups.

For example, lens distortion (barrel and pincushion) is modeled as aquadratic transformation, with barrel and pincushion having oppositeeffects. If this distortion is applied proportionally to depth, theresult is an image that has pincushion distortion for near objects andbarrel distortion for far objects, or vice versa. Other distortions,such as perspective distortion, can be applied in this depth dependentmanner as well.

Image Examples

FIGS. 5 and 6 show examples 500, 600 of opposite extremes of adolly-zoom action according to some embodiments.

FIGS. 7 and 8 show examples 700, 800 of opposite extremes of adolly-twirl action according to some embodiments.

Image Data Acquisition Devices

FIG. 9 is a block diagram of an architecture 900 of a light fieldcapture device such as a camera according to some embodiments. The lightfield capture device is used to implement some embodiments of thetechniques described herein. In the illustrated embodiment, the lightfield capture device includes a light field image data acquisitiondevice 905 that is configured to capture images using optics 910, animage sensor 915, which is implemented using a plurality of individualsensors for capturing pixels, and a microlens array 920. Someembodiments of the optics 910 include an aperture 925 for allowing aselectable amount of light into the light field capture device and amain lens 930 for focusing light toward the microlens array 920. Someembodiments of the microlens array 920 are disposed or incorporated inthe optical path of the light field capture device so as to facilitateacquisition, capture, sampling of, recording, or obtaining light fieldimage data via the sensor 915.

The light field image data acquisition device 905 includes the userinterface 935 for allowing a user to provide input for controlling theoperation of the light field capture device for capturing, acquiring,storing, or processing image data. Control circuitry 940 is used tofacilitate acquisition, sampling, recording, or obtaining light fieldimage data. For example, the control circuitry 940 can manage or control(automatically or in response to user input) the acquisition time and,rated acquisition, sampling, capturing, recording, or obtaining lightfield image data. Memory 945 is used to store image data, such as outputfrom the image sensor 915. The memory 945 is implemented as external orinternal memory, which can be provided as a separate device or locationrelative to the light field capture device. For example, the light fieldcapture device can store raw light field image data output by the imagesensor 915, or a representation thereof, such as a compressed image datafile.

Post-processing circuitry 950 in the light field image data acquisitiondevice 905 is used to access or modify image data acquired by the imagesensor 915. Some embodiments of the post-processing circuitry 950 areconfigured to create dolly zoom effects using the image data, asdiscussed herein. For example, the post-processing circuitry 950 caninclude one or more processors executing software stored in the memory945 to access and modify image data stored in the memory 945 to producedolly zoom effects in images captured by the image sensor 915.

FIG. 10 is a block diagram of an architecture 1000 of a light fieldcapture system according to some embodiments. The light field capturedevice is used to implement some embodiments of the techniques describedherein. Elements in FIG. 10 that are referenced using the same referencenumerals as corresponding elements in FIG. 9 perform the same or similarfunctions as the corresponding elements in FIG. 9. The architecture 1000shown in FIG. 10 differs from the architecture 900 shown in FIG. 9because the light field image data acquisition device 905 is implementedseparately from a postprocessing system 1005, which includes the memory945, the post-processing circuitry 950, and the user interface 935. Thelight field image data acquisition device 905 provides image data 1010to the postprocessing system 1005.

Refocus Analogies

The above-described techniques may, by analogy, be applied to lightfield refocus as well. For example, the refocus analogy of the dollytwirl effect would be:

$\begin{bmatrix}s^{\prime} \\t^{\prime} \\u^{\prime} \\v^{\prime \;}\end{bmatrix} = {\begin{bmatrix}1 & 0 & 0 & \phi \\0 & 1 & {- \phi} & 0 \\0 & 0 & 1 & 0 \\0 & 0 & 0 & 1\end{bmatrix}\begin{bmatrix}s \\t \\u \\v\end{bmatrix}}$

What is claimed is:
 1. A method comprising: determining four-dimensional(4D) coordinates of points in a light field; generating dolliedcoordinates from the 4D coordinates based on a dolly transform and adolly parameter; and projecting rays associated with the dolliedcoordinates from the light field onto corresponding points in an outputraster.
 2. The method of claim 1, further comprising: applying anaperture function to filter the rays in the coordinate system of thedollied coordinates.
 3. The method of claim 2, wherein the aperturefunction has a first value in a first region of an aperture plane andthe aperture value has a second value in a second region of the apertureplane such that rays passing through the first region are accepted bythe aperture function and rays passing through the second region arerejected.
 4. The method of claim 3, wherein applying the aperturefunction to filter the rays in the coordinate system of the dolliedcoordinates comprises dollying a virtual aperture by a distance relativeto the aperture plane.
 5. The method of claim 4, wherein generating thedollied coordinates comprises applying a depth dependent dollytransformation to the 4D coordinates.
 6. The method of claim 5, whereindollying the virtual aperture produces a depth—dependent scaling of atwo dimensional (2D) reconstructed image that is reconstructed from thelight field.
 7. The method of claim 1, further comprising: refocusingportions of the light field using a linear refocusing transform and arefocusing parameter.
 8. The method of claim 1, wherein generating thedollied coordinates comprises generating the dollied coordinates byapplying a quadratic dolly transform.
 9. The method of claim 1, whereingenerating the dollied coordinates comprises producing tilt in a lensplane and an aperture plane.
 10. An apparatus comprising: a sensorconfigured to acquire a light field by imaging a scene; and a processorconfigured to: determine four-dimensional (4D) coordinates of points ina light field; generate dollied coordinates from the 4D coordinatesbased on a dolly transform and a dolly parameter; and project raysassociated with the dollied coordinates from the light field ontocorresponding points in an output raster.
 11. The apparatus of claim 10,wherein the processor is configured to apply an aperture function tofilter the rays in the coordinate system of the dollied coordinates. 12.The apparatus of claim 11, wherein the aperture function has a firstvalue in a first region of an aperture plane and the aperture value hasa second value in a second region of the aperture plane such that rayspassing through the first region are accepted by the aperture functionand rays passing through the second region are rejected.
 13. Theapparatus of claim 12, wherein the processor is configured to dolly avirtual aperture by a distance relative to the aperture plane.
 14. Theapparatus of claim 13, wherein the processor is configured to apply adepth dependent dolly transformation to the 4D coordinates.
 15. Theapparatus of claim 14, wherein dollying the virtual aperture produces adepth-dependent scaling of a two dimensional (2D) reconstructed imagethat is reconstructed from the light field.
 16. The apparatus of claim10, wherein the processor is configured to refocus portions of the lightfield using a linear refocusing transform and a refocusing parameter.17. The apparatus of claim 10, wherein the processor is configured togenerate the dollied coordinates by applying a quadratic dollytransform.
 18. The apparatus of claim 10, wherein the processor isconfigured to tilt an image in a lens plane and an aperture planeapplying the dolly transform.